Learning Dexterous Manipulation Policies from Experience and Imitation

We explore learning-based approaches for feedback control of a dexterous five-finger hand performing non-prehensile manipulation. First we learn local controllers that are able to perform the task starting at a predefined initial state. These controllers are constructed using trajectory optimization with respect to locally-linear time-varying models learned directly from sensor data. In some cases we initialize the optimizer with human demonstrations collected via tele-operation in a virtual environment. We demonstrate that such controllers can perform the task robustly, both in simulation and on the physical platform, for a limited range of initial conditions around the trained starting state. We then consider two interpolation methods for generalizing to a wider range of initial conditions: deep learning, and nearest neighbours. We find that nearest neighbors achieve higher performance. Nevertheless the neural network has its advantages: it uses only tactile and proprioceptive feedback but no visual feedback about the object (i.e. it performs the task blind), and learns a time-invariant policy. In contrast, the nearest neighbours method switches between time-varying local controllers based on proximity of initial object states sensed via motion capture. While both generalization methods leave room for improvement, our work shows that (i) local trajectory-based controllers for complex non-prehensile manipulation tasks can be constructed from surprisingly small amounts of training data, and (ii) collections of such controllers can be interpolated to form more global controllers.